Depth estimation for stereo image pairs

Philips has developed a multi-view autostereoscopic 3D television which is capable of displaying video in real 3D without the use of external glasses or any other devices. Creating 3D content for these screens is difficult because they require not only the video itself but also the depth map as input. Depth maps are computed from a multi-view recording of the video sequence. To extract the depth map we have to solve the so called stereo correspondence problem. We developed a model, which with the help of some additional human input, can solve it reasonably well. We started by casting the original problem into the form of an energy minimization problem. The energy of a depth map consists of a weighted combination of two terms, a data term which indicates how well the depth maps corresponds with the input images, and an interaction term which is a measure of smoothness. The problem is then converted into the maximization of the probability of a Markov Random Field (MRF), a much used concept in the field of Computer Vision. An iterative procedure to obtain an approximate solution is derived from a Varia-tional Mean Field approximation of the MRF. Several extensions are made to the resulting model in order to make it more suitable for the difficulties that arise in practical situations. These include a multi-scale approach, a gradient based automatic boundary detection algorithm, the possibility to incorporate (manually selected) occluded and boundary pixels in the computation of the depth map and the option of using input from three camera views instead of just two. Our implementation of the model allows for several ways of giving manual input in order to obtain improved depth maps, of which the most important is the manual correction of the automatically computed boundaries. Unclassified Conclusions: In this report we have described a model for computing a depth map from a set of stereo images. We started with a mathematically sound model by casting the problem in the form of an energy minimization problem. This minimization problem is not exactly solvable and an iterative procedure was developed in order to obtain an approximate solution. While this solution method gives reasonable results, we made several extensions to the model in order to overcome problems that present themselves in real world example images. An important guideline in adding these extensions was that we wanted our model to accept user input in …